Voltage Control-Based Ancillary Service Using Deep Reinforcement Learning

Ancillary services rely on operating reserves to support an uninterrupted electricity supply that meets demand. One of the hidden grid is in thermostatically controlled loads. To efficiently exploit these reserves, a new realization control voltage allowable range follow set power reference proposed. The proposed approach based deep reinforcement learning (RL) algorithm. Double DQN utilized because proven state-of-the-art level performance complex tasks, native handling continuous environment state variables, and model-free application trained DDQN real grid. evaluate RL performance, method was compared with classic proportional change according setup. solution validated setups different number loads (TCLs) feeder show its generalization capabilities. In this article, particularities system domain are discussed along results achieved by such RL-powered demand response solution. tuning hyperparameters for algorithm performed achieve best double Q-network (DDQN) particular, influence rate, target network update step, layer size, batch replay buffer size were assessed. roughly two times better than competing optimal selection within considered time interval simulation. decrease deviation actual consumption from profile demonstrated. benefit costs estimated presented control-based ancillary service potential impact.